{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Review  - SciPy and Scikit-Learn\n",
    "\n",
    "---\n",
    "- Materials in this module include resources collected from various open-source online repositories.\n",
    "- Jupyter source file can be downloaded from https://github.com/gaoshangdeakin/SIT384-Jupyter\n",
    "- If you found any issue/bug for this document, please submit an issue at [https://github.com/gaoshangdeakin/SIT384/issues](https://github.com/gaoshangdeakin/SIT384/issues)\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "**The purpose of this session is to illustrate**\n",
    "\n",
    "1. How to test installed Numpy and Scikit-Learn packages\n",
    "2. How to use scikit-learn to load datasets and examine their properties, getting ready for ML tasks.\n",
    "\n",
    "** References and additional reading and resources**\n",
    "- [Installing scikit-learn](http://scikit-learn.org/stable/install.html)\n",
    "- [Lectures on scientific computing with Python](https://github.com/jrjohansson/scientific-python-lectures)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Scipy: Scientific Computing Tools for Python\n",
    "\n",
    "<!--span style=\"color:#0b486b\">3. Scipy: Scientific Computing Tools for Python</span-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[SciPy](https://www.scipy.org) (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics, science, and engineering. In particular, the SciPy ecosystem these are some of the core packages:\n",
    "\n",
    "- Numpy: a base N-dimensional array package\n",
    "- Matplotlib: a package that allows you to create charts and plots from data.\n",
    "- Pandas: tools and data structures to organize and analyze your data.\n",
    "- SciPy library: a fundamental library for scientific computing such as optimization.\n",
    "\n",
    "All of these packages are pre-installed when you install the Anaconda. You can check the availibility and versions of above packages as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The scipy version is 1.0.0\n",
      "The numpy version is 1.13.3.\n",
      "The matplotlib version is 2.1.0\n",
      "The pandas version is 0.21.0\n"
     ]
    }
   ],
   "source": [
    "# scipy\n",
    "import scipy\n",
    "print('The scipy version is {}'.format(scipy.__version__))\n",
    "\n",
    "# numpy\n",
    "import numpy\n",
    "print(\"The numpy version is {}.\".format(numpy.__version__))\n",
    "\n",
    "# matplotlib\n",
    "import matplotlib\n",
    "print('The matplotlib version is {}'.format(matplotlib.__version__))\n",
    "\n",
    "# pandas\n",
    "import pandas\n",
    "print('The pandas version is {}'.format(pandas.__version__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now briefly go through these packages to know NumPy, Matplotlib and Pandas to read and write machine learning Python\n",
    "scripts."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numpy<!--3.1 Numpy-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Numpy](http://www.numpy.org/) is the core library for scientific computing in Python. It provides a high-performance multidimensional array object, and tools for working with these arrays. Numpy is a pre-built package included in Anaconda 3. You only need to import to use the package."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "###  Arrays<!--3.1.1 Arrays-->\n",
    "A numpy array is a grid of values, all of the same type, and is indexed by a tuple of nonnegative integers. The number of dimensions is the <em>rank</em> of the array; the <em>shape</em> of an array is a tuple of integers giving the size of the array along each dimension.\n",
    "You can initialize numpy arrays from nested Python lists or use pre-defined functions to create arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Arrays are created from nested Python lists\n",
      "\n",
      "The data type of a is <class 'numpy.ndarray'>\n",
      "The shape of a is (3,).\n",
      "The shape of a is (2, 3).\n",
      "\n",
      "Arrays are created with pre-defined functions\n",
      "\n",
      "An all-zero array\n",
      "\n",
      "[[ 0.  0.]\n",
      " [ 0.  0.]\n",
      " [ 0.  0.]]\n",
      "\n",
      "An all ones array\n",
      "\n",
      "[[ 1.  1.]\n",
      " [ 1.  1.]\n",
      " [ 1.  1.]]\n",
      "\n",
      "A constant \n",
      "\n",
      "[[7 7]\n",
      " [7 7]]\n",
      "\n",
      "A 2x2 identity  array\n",
      "\n",
      "[[ 1.  0.]\n",
      " [ 0.  1.]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "a = np.array([1, 2, 3])   # Create a rank 1 array\n",
    "b = np.array([[1,2,3],[4,5,6]])    # Create a rank 2 array\n",
    "\n",
    "print(\"Arrays are created from nested Python lists\\n\")\n",
    "\n",
    "print(\"The data type of a is {}\".format(type(a)))          \n",
    "print(\"The shape of a is {}.\".format(a.shape))    \n",
    "\n",
    "print(\"The shape of a is {}.\".format(b.shape))               \n",
    "\n",
    "print(\"\\nArrays are created with pre-defined functions\\n\")\n",
    "\n",
    "c = np.zeros((3,2))   # Create an array of all zeros\n",
    "print(\"An all-zero array\\n\")\n",
    "print(c)           \n",
    "\n",
    "d = np.ones((3,2))    # Create an array of all ones\n",
    "print(\"\\nAn all ones array\\n\")\n",
    "print(d)\n",
    "\n",
    "e = np.full((2,2), 7)  # Create a constant array\n",
    "print(\"\\nA constant \\n\")\n",
    "print(e)\n",
    "\n",
    "f = np.eye(2)         # Create a 2x2 identity matrix\n",
    "print(\"\\nA 2x2 identity  array\\n\")\n",
    "print(f)      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Array indexing<!--3.1.2  Array indexing-->\n",
    "Numpy offers several ways to index into arrays.\n",
    "\n",
    "<strong>Slicing:</strong>Numpy arrays can be sliced. Since arrays may be multidimensional, you must specify a slice for each dimension of the array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "77\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])\n",
    "\n",
    "# Slice the subarray consisting of the first 2 rows and columns 1 and 2; \n",
    "b = a[:2, 1:3]\n",
    "\n",
    "# A slice of an array is a view into the same data, so modifying it\n",
    "# will modify the original array.\n",
    "\n",
    "print(a[0, 1])   \n",
    "b[0, 0] = 77     # b[0, 0] is the same piece of data as a[0, 1]\n",
    "print(a[0, 1])   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also mix integer indexing with slice indexing to  yield an array of lower rank than the original array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5 6 7 8] (4,)\n",
      "[77  6 10] (3,)\n"
     ]
    }
   ],
   "source": [
    "row_r1 = a[1, :]    # Rank 1 view of the second row of a\n",
    "print(row_r1, row_r1.shape) \n",
    "\n",
    "# We can make the same distinction when accessing columns of an array:\n",
    "col_r1 = a[:, 1]\n",
    "print(col_r1, col_r1.shape)  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<strong>Integer array indexing:</strong> When you index into numpy arrays using slicing, the resulting array view will always be a subarray of the original array. In contrast, integer array indexing allows you to construct arbitrary arrays using the data from another array. Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1 77  3  4]\n",
      " [ 5  6  7  8]\n",
      " [ 9 10 11 12]]\n",
      "[1 6 9]\n",
      "[1 6 9]\n",
      "[77 77]\n",
      "[77 77]\n"
     ]
    }
   ],
   "source": [
    "print(a) \n",
    "\n",
    "\n",
    "# The returned array will have shape (3,) by choosing elements \n",
    "# at [0,1],[1,1], and [2,0]\n",
    "print(a[[0, 1, 2], [0, 1, 0]])\n",
    "\n",
    "# The above example of integer array indexing is equivalent to this:\n",
    "print(np.array([a[0, 0], a[1, 1], a[2, 0]]))\n",
    "\n",
    "# When using integer array indexing, you can reuse the same\n",
    "# element from the source array:\n",
    "print(a[[0, 0], [1, 1]])  # Prints \"[2 2]\"\n",
    "\n",
    "# Equivalent to the previous integer array indexing example\n",
    "print(np.array([a[0, 1], a[0, 1]])) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<strong>Boolean array indexing:</strong> Boolean array indexing lets you pick out arbitrary elements of an array. Frequently this type of indexing is used to select the elements of an array that satisfy some condition. Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[False  True False False]\n",
      " [False  True  True  True]\n",
      " [ True  True  True  True]]\n",
      "[77  6  7  8  9 10 11 12]\n",
      "[77  6  7  8  9 10 11 12]\n"
     ]
    }
   ],
   "source": [
    "# Find the elements of a that are bigger than 2;this returns a \n",
    "# numpy array of Booleans of the same shape as a, where each \n",
    "# slot of bool_idx tells whether that element of a is > 5.\n",
    "\n",
    "bool_idx = (a > 5)  \n",
    "print(bool_idx)     \n",
    "\n",
    "# We use boolean array indexing to construct a rank 1 array consisting \n",
    "# of the elements of a corresponding to the True values of bool_idx\n",
    "print(a[bool_idx])  \n",
    "\n",
    "# We can do all of the above in a single concise statement:\n",
    "print(a[a > 5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can refer to [indexing documentation](https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html) for further details about numpy array indexing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Array math operations <!--3.1.3  Array math operations-->\n",
    "\n",
    "Basic mathematical functions operate elementwise on arrays, and are available both as operator overloads and as functions in the numpy module:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elementwise sum; both produce the array\n",
      "[[  6.   8.]\n",
      " [ 10.  12.]]\n",
      "[[  6.   8.]\n",
      " [ 10.  12.]]\n",
      "Elementwise difference; both produce the array\n",
      "[[-4. -4.]\n",
      " [-4. -4.]]\n",
      "[[-4. -4.]\n",
      " [-4. -4.]]\n",
      "Elementwise product; both produce the array\n",
      "[[  5.  12.]\n",
      " [ 21.  32.]]\n",
      "[[  5.  12.]\n",
      " [ 21.  32.]]\n",
      "Elementwise division; both produce the array\n",
      "[[ 0.2         0.33333333]\n",
      " [ 0.42857143  0.5       ]]\n",
      "[[ 0.2         0.33333333]\n",
      " [ 0.42857143  0.5       ]]\n",
      "Elementwise square root; produces the array\n",
      "[[ 1.          1.41421356]\n",
      " [ 1.73205081  2.        ]]\n"
     ]
    }
   ],
   "source": [
    "x = np.array([[1,2],[3,4]], dtype=np.float64)\n",
    "y = np.array([[5,6],[7,8]], dtype=np.float64)\n",
    "\n",
    "print(\"Elementwise sum; both produce the array\")\n",
    "print(x + y)\n",
    "print(np.add(x, y))\n",
    "\n",
    "print(\"Elementwise difference; both produce the array\")\n",
    "print(x - y)\n",
    "print(np.subtract(x, y))\n",
    "\n",
    "print(\"Elementwise product; both produce the array\")\n",
    "print(x * y)\n",
    "print(np.multiply(x, y))\n",
    "\n",
    "print(\"Elementwise division; both produce the array\")\n",
    "print(x / y)\n",
    "print(np.divide(x, y))\n",
    "\n",
    "print(\"Elementwise square root; produces the array\")\n",
    "print(np.sqrt(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To execute matrix multiplication,  We instead use the <kbd>dot</kbd> function to compute inner products of vectors, to multiply a vector by a matrix, and to multiply matrices. <kbd>dot</kbd> is available both as a function in the numpy module and as an instance method of array objects: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "219\n",
      "219\n",
      "[ 29.  67.]\n",
      "[[ 19.  22.]\n",
      " [ 43.  50.]]\n"
     ]
    }
   ],
   "source": [
    "v = np.array([9,10])\n",
    "w = np.array([11, 12])\n",
    "\n",
    "# Inner product of vectors; both produce 219\n",
    "print(v.dot(w))\n",
    "print(np.dot(v, w))\n",
    "\n",
    "# Matrix / vector product; both produce the rank 1 array [29 67]\n",
    "print(x.dot(v))\n",
    "\n",
    "# print(np.dot(x, v))\n",
    "\n",
    "# Matrix / matrix product; both produce the rank 2 array\n",
    "print(x.dot(y))\n",
    "#print(np.dot(x, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy provides many useful functions for performing computations on arrays; Some useful function are <kbd>sum</kbd>, transpose <kbd>T</kbd>. The full list of mathematical functions provided by numpy in [the documentation](https://docs.scipy.org/doc/numpy/reference/routines.math.html). More functions for manipulating arrays can be found in [the documentation](https://docs.scipy.org/doc/numpy/reference/routines.array-manipulation.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10\n",
      "[4 6]\n",
      "[3 7]\n",
      "[[1 3]\n",
      " [2 4]]\n"
     ]
    }
   ],
   "source": [
    "x = np.array([[1,2],[3,4]])\n",
    "print(np.sum(x))  # Compute sum of all elements\n",
    "print(np.sum(x, axis=0))  # Compute sum of each column\n",
    "print(np.sum(x, axis=1))  # Compute sum of each row\n",
    "\n",
    "print(x.T)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Broadcasting <!--3.1.4 Broadcasting-->\n",
    "Broadcasting is a powerful mechanism that allows numpy to work with arrays of different shapes when performing arithmetic operations. Frequently we have a smaller array and a larger array, and we want to use the smaller array multiple times to perform some operation on the larger array.\n",
    "For example, suppose that we want to add a constant vector to each row of a matrix. We could do it like this:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 2  2  4]\n",
      " [ 5  5  7]\n",
      " [ 8  8 10]\n",
      " [11 11 13]]\n"
     ]
    }
   ],
   "source": [
    "x = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])\n",
    "v = np.array([1, 0, 1])\n",
    "\n",
    "y=x+v\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Broadcasting two arrays together follows these rules:\n",
    "\n",
    "1. If the arrays do not have the same rank, prepend the shape of the lower rank array with 1s until both shapes have the same length.\n",
    "2. The two arrays are said to be compatible in a dimension if they have the same size in the dimension, or if one of the arrays has size 1 in that dimension.\n",
    "3. The arrays can be broadcast together if they are compatible in all dimensions.\n",
    "4. After broadcasting, each array behaves as if it had shape equal to the elementwise maximum of shapes of the two input arrays.\n",
    "5. In any dimension where one array had size 1 and the other array had size greater than 1, the first array behaves as if it were copied along that dimension\n",
    "\n",
    "More explanation for broadcasting arrays can be found in [the documentation](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matplotlib <!--3.2 Matplotlib-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matplotlib can be used for creating plots and charts. The library is generally used as follows:\n",
    "- Call a plotting function with some data (e.g. .plot()).\n",
    "- Call many functions to setup the properties of the plot (e.g. labels and colors).\n",
    "- Make the plot visible (e.g. .show()).\n",
    "\n",
    "You can create a simple line plot from one dimensional data as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2b1b13619400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# basic line plot\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy\n",
    "myarray = numpy.array([1, 2, 3])\n",
    "plt.plot(myarray)\n",
    "plt.xlabel('X axis')\n",
    "plt.ylabel('Y axis')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You also can creat a scatter plot from two dimensional data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAF1tJREFUeJzt3X/wXXV95/HnqyFCFDAiXzWEQHRk6IiI0O8illbxxxpECqxiS3erwuimWl1118URu6srnY7jsrXqOoVGbQsWKwwCRRZButZR6wB+Q4CAwJpFKkno8BUMEMkiie/9454cv1zu9xfk3JuQ52Pmzj3ncz733nfOnNzX9/y455OqQpIkgF8bdQGSpJ2HoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqTWHqMuYL7233//Wr58+ajLkKRdyurVq39aVWOz9dvlQmH58uVMTEyMugxJ2qUk+ee59PPwkSSpZShIklqGgiSpZShIklqGgiSp1WkoJFmc5JIkdyS5Pckr+5YnyeeSrEtyS5KjuqxHkjSzri9J/SxwdVWdmuQZwDP7lr8ROKR5vAI4t3mWpN3e5Ws2cM41d7Jx0xYOWLyIM1ccyilHLu30MzsLhST7Aq8CTgeoql8Av+jrdjJwQfXGBL2u2bNYUlX3dlWXJO0KLl+zgbMuXcuWx7YBsGHTFs66dC1Ap8HQ5eGjFwGTwF8nWZPki0me1ddnKXDPlPn1TZsk7dbOuebONhC22/LYNs655s5OP7fLUNgDOAo4t6qOBH4OfKSvTwa8rvobkqxMMpFkYnJycsdXKkk7mY2btsyrfUfpMhTWA+ur6vpm/hJ6IdHfZ9mU+QOBjf1vVFWrqmq8qsbHxma9dYck7fIOWLxoXu07SmehUFX/AtyT5NCm6XXAD/u6XQG8vbkK6RjgQc8nSBKcueJQFi1c8Li2RQsXcOaKQ6d5xY7R9dVH/wG4sLny6C7gjCTvBqiq84CrgBOAdcAjwBkd1yNJu4TtJ5OHffVRehf+7DrGx8fLu6RK0vwkWV1V47P18xfNkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJahkKkqSWoSBJanU68lqSu4GHgW3A1v4BHpIcB/w98OOm6dKqOrvLmiRJ0+t6OE6A11TVT2dY/t2qOnEIdUiSZuHhI0lSq+tQKOCbSVYnWTlNn1cmuTnJN5IcNqhDkpVJJpJMTE5OdletJO3muj58dGxVbUzyPODaJHdU1XemLL8ROLiqNic5AbgcOKT/TapqFbAKYHx8vDquWZJ2W53uKVTVxub5PuAy4Oi+5Q9V1eZm+ipgYZL9u6xJkjS9zkIhybOS7LN9GngDcGtfnxckSTN9dFPP/V3VJEmaWZeHj54PXNZ85+8BfKWqrk7yboCqOg84FXhPkq3AFuC0qvLwkCSNSGehUFV3AUcMaD9vyvTngc93VYMkaX68JFWS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1Oo0FJLcnWRtkpuSTAxYniSfS7IuyS1JjuqyHknSzLoceW2711TVT6dZ9kbgkObxCuDc5lmSNAKjPnx0MnBB9VwHLE6yZMQ1SdJuq+tQKOCbSVYnWTlg+VLgninz65s2SdIIdH346Niq2pjkecC1Se6oqu9MWZ4Br6n+hiZQVgIcdNBB3VQqSep2T6GqNjbP9wGXAUf3dVkPLJsyfyCwccD7rKqq8aoaHxsb66pcSdrtdRYKSZ6VZJ/t08AbgFv7ul0BvL25CukY4MGqurermiRJM+vy8NHzgcuSbP+cr1TV1UneDVBV5wFXAScA64BHgDM6rEeSNIvOQqGq7gKOGNB+3pTpAt7bVQ2SpPkZ9SWpkqSdiKEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWoZCpKklqEgSWp1HgpJFiRZk+TKActOTzKZ5Kbm8a6u65EkTa/L4Ti3+wBwO7DvNMsvqqr3DaEOSdIsOt1TSHIg8Cbgi11+jiRpx+j68NFngA8Dv5yhz1uS3JLkkiTLOq5HkjSDzkIhyYnAfVW1eoZuXweWV9XLgH8Azp/mvVYmmUgyMTk52UG1kiTodk/hWOCkJHcDXwVem+Rvp3aoqvur6tFm9gvAbwx6o6paVVXjVTU+NjbWYcmStHvrLBSq6qyqOrCqlgOnAd+qqj+Y2ifJkimzJ9E7IS1JGpFhXH30OEnOBiaq6grg/UlOArYCDwCnD7seSdKvpKpGXcO8jI+P18TExKjLkKRdSpLVVTU+Wz9/0SxJahkKkqSWoSBJahkKkqSWoSBJahkKkqTWrKGQ5K1J9mmm/0uSS5Mc1X1pkqRhm8uewn+tqoeT/Bawgt79ic7ttixJ0ijMJRS2Nc9vAs6tqr8HntFdSZKkUZlLKGxI8pfA7wJXJdlzjq+TJO1i5vLl/rvANcDxVbUJ2A84s9OqJEkjMe0N8ZLsW1UPAXsB327a9gMeBbz5kCQ9Dc10l9SvACcCq4ECMmVZAS/qsC5J0ghMGwpVdWLz/MLhlSNJGqW5/E7hnX3zC5J8vLuSJEmjMpcTza9LclWSJUkOB64D9pnrBzQhsibJlQOW7ZnkoiTrklyfZPmcK5ck7XCzjrxWVf82ye8Ba4FHgN+vqn+ax2d8gN4wm/sOWPZO4GdV9eIkpwGfAn5vHu8t7RQuX7OBc665k42btnDA4kWcueJQTjly6ajLkuZtLoePDqH3xf414G7gbUmeOZc3T3IgvR+9fXGaLifT+4U0wCX09koyTV9pp3T5mg2cdelaNmzaQgEbNm3hrEvXcvmaDaMuTZq3uRw++jq9W138IfBq4EfAD+b4/p8BPgz8cprlS4F7AKpqK/Ag8Nw5vre0UzjnmjvZ8ti2x7VteWwb51xz54gqkp68uYTC0VX1vwGq58+AU2Z7UZITgfuqavVM3Qa0PWHQ6CQrk0wkmZicnJxDydLwbNy0ZV7t0s5sLucUHkryUuAl9H7Itt2PZnnpscBJSU5oXrdvkr+tqj+Y0mc9sAxYn2QP4NnAAwNqWAWsAhgfH39CaEijdMDiRWwYEAAHLF40gmqkp2Yu5xQ+DvzP5vEa4L8DJ832uqo6q6oOrKrlwGnAt/oCAeAK4B3N9KlNH7/0tUs5c8WhLFq44HFtixYu4MwVh46oIunJm8vho1OB1wH/UlVnAEcAez7ZD0xydpLtofIl4LlJ1gH/CfjIk31faVROOXIpn3zz4SxdvIgASxcv4pNvPtyrj7RLymx/mCe5oaqOTrKa3p7Cw8CtVXXYMArsNz4+XhMT3npJkuYjyeqqGp+t36znFICJJIuBL9C7D9Jm4IanWJ8kaSc0lxPNf9RMnpfkamDfqrql27IkSaMwlz2FVlXd3VEdkqSdgCOoSZJa04ZCcxO85cMrRZI0ajPtKfwN8M0kf5xk4ZDqkSSN0EyD7Fyc5H8BH6N3BdKXmXIPo6r69BDqkyQN0Wwnmh8Dfk7vx2r7MP2N7SRJTwPThkKS44FP07sVxVFV9cjQqpIkjcRMewp/DLy1qm4bVjGSpNGa6ZzCbw+zEEnS6Pk7BUlSy1CQJLUMBUlSy1CQJLUMBUlSq7NQSLJXkhuS3JzktiSfGNDn9CSTSW5qHu/qqh5J0uzmdevseXoUeG1VbW7unfS9JN+oquv6+l1UVe/rsA5J0hx1FgrVG+dzczO7sHnMPPanJGmkOj2nkGRBkpuA+4Brq+r6Ad3ekuSWJJckWdZlPZKkmXUaClW1rapeDhwIHJ3kpX1dvg4sr6qXAf8AnD/ofZKsTDKRZGJycrLLkiVptzaUq4+qahPwbeD4vvb7q+rRZvYLwG9M8/pVVTVeVeNjY2Od1ipJu7Murz4aS7K4mV4EvB64o6/PkimzJwG3d1WPJGl2XV59tAQ4P8kCeuFzcVVdmeRsYKKqrgDen+QkYCvwAHB6h/VIkmaR3kVCu47x8fGamJgYdRmStEtJsrqqxmfr5y+aJUktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEktQ0GS1DIUJEmtLkde2yvJDUluTnJbkk8M6LNnkouSrEtyfZLlXdUjSZpdl3sKjwKvraojgJcDxyc5pq/PO4GfVdWLgT8HPtVhPZKkWXQWCtWzuZld2Dz6h3k7GTi/mb4EeF2SdFWTJGlmnZ5TSLIgyU3AfcC1VXV9X5elwD0AVbUVeBB4bpc1SZKm12koVNW2qno5cCBwdJKX9nUZtFfwhEGjk6xMMpFkYnJysotSJUkM6eqjqtoEfBs4vm/RemAZQJI9gGcDDwx4/aqqGq+q8bGxsY6rlaTdV5dXH40lWdxMLwJeD9zR1+0K4B3N9KnAt6rqCXsKkqTh2KPD914CnJ9kAb3wubiqrkxyNjBRVVcAXwK+nGQdvT2E0zqsR5I0i85CoapuAY4c0P6xKdP/D3hrVzVIkubHXzRLklqGgiSpZShIklqGgiSpZShIklqGgiSpZShIklqGgiSpZShIklqGgiSpZShIklqGgiSpZShIklqGgiSpZShIklqGgiSp1eVwnMuS/GOS25PcluQDA/ocl+TBJDc1j48Nei9J0nB0ORznVuBDVXVjkn2A1Umuraof9vX7blWd2GEdkqQ56mxPoaruraobm+mHgduBpV19niTpqRvKOYUky+mN13z9gMWvTHJzkm8kOWwY9UiSBuvy8BEASfYGvgZ8sKoe6lt8I3BwVW1OcgJwOXDIgPdYCawEOOiggzquWJJ2X53uKSRZSC8QLqyqS/uXV9VDVbW5mb4KWJhk/wH9VlXVeFWNj42NdVmyJO3Wurz6KMCXgNur6tPT9HlB048kRzf13N9VTZKkmXV5+OhY4G3A2iQ3NW0fBQ4CqKrzgFOB9yTZCmwBTquq6rAmSdIMOguFqvoekFn6fB74fFc1SJLmx180S5JahoIkqWUoSJJahoIkqWUoSJJahoIkqWUoSJJahoIkqWUoSJJahoIkqWUoSJJahoIkqWUoSJJahoIkqWUoSJJahoIkqdXZIDtJlgEXAC8AfgmsqqrP9vUJ8FngBOAR4PSqurGrmi5fs4FzrrmTjZu2cMDiRZy54lBOOXJpVx8nSbucLofj3Ap8qKpuTLIPsDrJtVX1wyl93ggc0jxeAZzbPO9wl6/ZwFmXrmXLY9sA2LBpC2dduhbAYJCkRmeHj6rq3u1/9VfVw8DtQP+378nABdVzHbA4yZIu6jnnmjvbQNhuy2PbOOeaO7v4OEnaJQ3lnEKS5cCRwPV9i5YC90yZX88Tg4MkK5NMJJmYnJx8UjVs3LRlXu2StDvqPBSS7A18DfhgVT3Uv3jAS+oJDVWrqmq8qsbHxsaeVB0HLF40r3ZJ2h11GgpJFtILhAur6tIBXdYDy6bMHwhs7KKWM1ccyqKFCx7XtmjhAs5ccWgXHydJu6TOQqG5suhLwO1V9elpul0BvD09xwAPVtW9XdRzypFL+eSbD2fp4kUEWLp4EZ988+GeZJakKbq8+uhY4G3A2iQ3NW0fBQ4CqKrzgKvoXY66jt4lqWd0WA+nHLnUEJCkGXQWClX1PQafM5jap4D3dlWDJGl+/EWzJKllKEiSWoaCJKllKEiSWoaCJKmV3gVAu44kk8A/P8W32R/46Q4oZ0faGWsC65qvnbGunbEmsK752BE1HVxVs94SYpcLhR0hyURVjY+6jql2xprAuuZrZ6xrZ6wJrGs+hlmTh48kSS1DQZLU2l1DYdWoCxhgZ6wJrGu+dsa6dsaawLrmY2g17ZbnFCRJg+2uewqSpAGeVqGQ5K+S3Jfk1mmWJ8nnkqxLckuSo6Yse0eSHzWPdwyxpn/X1HJLku8nOWLKsruTrE1yU5KJHVXTHOs6LsmDzWfflORjU5Ydn+TOZj1+ZMh1nTmlpluTbEuyX7Osk/WVZFmSf0xye5LbknxgQJ9RbFtzqWvo29cc6xrq9jXHmkaxbe2V5IYkNzd1fWJAnz2TXNSsj+vTG8ly+7KzmvY7k6zYIUVV1dPmAbwKOAq4dZrlJwDfoHf31mOA65v2/YC7mufnNNPPGVJNv7n9s4A3bq+pmb8b2H9E6+o44MoB7QuA/wu8CHgGcDPwkmHV1df3d4Bvdb2+gCXAUc30PsD/6f83j2jbmktdQ9++5ljXULevudQ0om0rwN7N9EJ6QxYf09fnj4DzmunTgIua6Zc062dP4IXNelvwVGt6Wu0pVNV3gAdm6HIycEH1XAcsTrIEWAFcW1UPVNXPgGuB44dRU1V9v/lMgOvojT7XuTmsq+kcDayrqruq6hfAV+mt11HU9fvA3+2oz55OVd1bVTc20w8Dt/PEscRHsW3NWtcotq85rq/pdLJ9PYmahrVtVVVtbmYXNo/+E70nA+c305cAr0uSpv2rVfVoVf2Y3rg0Rz/Vmp5WoTAHS4F7psyvb9qmax+2d9L7a3O7Ar6ZZHWSlSOo55XNbu03khzWtO0U6yrJM+l9uX5tSnPn66vZdT+S3l90U41025qhrqmGvn3NUtdItq/Z1tWwt60kC9IbiOw+en9ATLttVdVW4EHguXS0rroceW1nNGjQn5qhfWiSvIbef9rfmtJ8bFVtTPI84NokdzR/SQ/DjfR+Fr85yQnA5cAh7ATrqvE7wD9V1dS9ik7XV5K96X1RfLCqHupfPOAlQ9m2Zqlre5+hb1+z1DWS7Wsu64ohb1tVtQ14eZLFwGVJXlpVU8+pDXXb2t32FNYDy6bMHwhsnKF9KJK8DPgicHJV3b+9vao2Ns/3AZexA3YN56qqHtq+W1tVVwELk+zPiNfVFKfRt3vf5fpKspDel8mFVXXpgC4j2bbmUNdItq/Z6hrF9jWXddUY6rY15TM2Ad/miYcX23WSZA/g2fQOsXazrnbUCZOd5QEsZ/qTp2/i8ScDb2ja9wN+TO9E4HOa6f2GVNNB9I4F/mZf+7OAfaZMfx84fojr6gX86ncsRwM/adbbHvROlr6QX50IPGxYdTXLt/+neNYw1lfz774A+MwMfYa+bc2xrqFvX3Osa6jb11xqGtG2NQYsbqYXAd8FTuzr814ef6L54mb6MB5/ovkudsCJ5qfV4aMkf0fvqob9k6wHPk7vxA1VdR5wFb2rRNYBjwBnNMseSPInwA+atzq7Hr/r2GVNH6N3fPAveueO2Fq9G189n96uJPT+o3ylqq7eETXNsa5Tgfck2QpsAU6r3pa4Ncn7gGvoXSnyV1V12xDrAvg3wDer6udTXtrl+joWeBuwtjn2C/BRel+4I9u25ljXKLavudQ17O1rLjXB8LetJcD5SRbQO3JzcVVdmeRsYKKqrgC+BHw5yTp6gXVaU/NtSS4GfghsBd5bvUNRT4m/aJYktXa3cwqSpBkYCpKklqEgSWoZCpKklqEgSWoZClKf5o6aP55yh8znNPMH74D3/v5Tr1DqjpekSgMk+TDw4qpameQvgbur6pOjrkvqmnsK0mB/DhyT5IP07hf0Z4M6Jbm8uUnabdtvlJbk4PTGTtg/ya8l+W6SNzTLNjfPS5J8Z8q9+397SP8uaUbuKUjTaAYtuRp4Q1VdO02f/ZpfLS+i96vlV1fV/UneRe8eNtfT2+P4w6b/5qraO8mHgL2q6k+bX7M+s3q3dJZGyj0FaXpvBO4FXjpDn/cnuZneWAXL6N3pk6r6Ir3BXN4N/OcBr/sBcEaS/wYcbiBoZ2EoSAMkeTnwr+nd3O4/NgPm9Pc5Dng98MqqOgJYA+zVLHsmvxrQZu/+11bvtsuvAjbQu6/N2zv4Z0jzZihIfZpRrc6ld8/9nwDnAP9jQNdnAz+rqkeS/Dq9ANnuU8CF9G5I94UBn3EwcF9VfYHeDc+O6u8jjYKhID3Rvwd+MuU8wl8Av57k1X39rgb2SHIL8Cf0DiHR9PtXwKeq6kLgF0nO6HvtccBNSdYAbwE+28m/RJonTzRLklruKUiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKllKEiSWoaCJKn1/wG5p9Dnd6IX4wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2b1b40995b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# basic scatter plot\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy\n",
    "x = numpy.array([1, 2, 3])\n",
    "y = numpy.array([2, 4, 6])\n",
    "plt.scatter(x,y)\n",
    "plt.xlabel('X axis')\n",
    "plt.ylabel('Y axis')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many more plot types and many more properties that can be set on a plot. See [matplotlib for beginner](http://matplotlib.org/users/beginner.html) and [examples](https://matplotlib.org/examples/index.html) to learn more about the Matplotlib API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pandas <!--3.3 Pandas-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas provides data structures and functionality to quickly manipulate and analyze data. The\n",
    "key to understanding Pandas for machine learning is understanding the **Series** and **DataFrame**\n",
    "data structures."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Series <!--3.3.1 Series-->\n",
    "A series is a one dimensional array of data where the rows are labeled using a time axis. You can access the data in a series like a NumPy array or like a dictionary.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a    1\n",
      "b    2\n",
      "c    3\n",
      "dtype: int64\n",
      "1\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# series\n",
    "import numpy\n",
    "import pandas\n",
    "myarray = numpy.array([1, 2, 3])\n",
    "rownames = ['a', 'b', 'c']\n",
    "myseries = pandas.Series(myarray, index=rownames)\n",
    "print(myseries)\n",
    "\n",
    "print(myseries[0])\n",
    "print(myseries['a'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame <!--3.3.2 DataFrame-->\n",
    "A data frame is a multi-dimensional array where the rows and the columns can be labeled.A data frame can be indexed using column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   one  two  three\n",
      "a    1    2      3\n",
      "b    4    5      6\n",
      "method 1:\n",
      "one column:\n",
      "a    1\n",
      "b    4\n",
      "Name: one, dtype: int64\n",
      "method 2:\n",
      "one column:\n",
      "a    1\n",
      "b    4\n",
      "Name: one, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# dataframe\n",
    "import numpy\n",
    "import pandas\n",
    "myarray = numpy.array([[1, 2, 3], [4, 5, 6]])\n",
    "rownames = ['a', 'b']\n",
    "colnames = ['one', 'two', 'three']\n",
    "mydataframe = pandas.DataFrame(myarray, index=rownames, columns=colnames)\n",
    "print(mydataframe)\n",
    "\n",
    "print(\"method 1:\")\n",
    "print(\"one column:\\n%s\" % mydataframe['one'])\n",
    "\n",
    "print(\"method 2:\")\n",
    "print(\"one column:\\n%s\" % mydataframe.one)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Examples for manipulation with data using can be found at [Pandas cookbook](http://pandas.pydata.org/pandas-docs/stable/cookbook.html) which provides many short and nice examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scipy <!--3.4 Scipy-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optimization (finding minima or maxima of a function) is a package which is usual used in machine learning. Here we will only look at a few very simple cases.\n",
    "\n",
    "To use the optimization module in scipy first include the optimize module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import optimize\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first look at how to find the minima of a simple function of a single variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return 4*x**3 + (x-2)**2 + x**4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which can be plotted as "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD9CAYAAACyYrxEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4ZHWd7/H3t7JvlX3tpPd0p9NbGkKzIwhIgw6IXll0ENfWZ3DGba6K3hlx5jr6jNdtxhkUpQdwEERZFAcXRKBlbdL7vqezL519X6q+9486acs2TdKdqjqVyvf1PPUkdXIq9SHAJye/8zu/I6qKMcaY2OVxO4AxxpjwsqI3xpgYZ0VvjDExzoreGGNinBW9McbEOCt6Y4yJcVMWvYiUicjzIrJfRPaKyCed7Tki8qyIHHY+ZjvbRUT+TUSOiMguETkv3P8Qxhhjzmw6R/TjwGdVdQVwEXCXiFQCXwCeU9Vy4DnnOcD1QLnz2AjcG/LUxhhjpm3KolfVZlXd5nzeB+wH5gE3AQ86uz0IvNP5/CbgIQ14DcgSkeKQJzfGGDMtZzVGLyILgXXA60ChqjZD4JcBUODsNg+oD3pZg7PNGGOMC+Knu6OIpAOPA59S1V4ROeOuk2z7i3UWRGQjgaEd0tLSzq+oqJhuFGOMMcDWrVtPqmr+VPtNq+hFJIFAyT+sqk84m1tFpFhVm52hmTZnewNQFvTyUqDp9O+pqvcB9wFUV1drTU3NdKIYY4xxiMiJ6ew3nVk3AtwP7FfVbwV96ZfAnc7ndwK/CNr+fmf2zUVAz8QQjzHGmMibzhH9pcAdwG4R2eFs+yLwdeAxEfkwUAe8x/naM8ANwBFgEPhgSBMbY4w5K1MWvaq+xOTj7gBXT7K/AnfNMJcxxpgQsStjjTEmxlnRG2NMjLOiN8aYGGdFb4wxMc6K3hhjXLLppeM8szv8s8+t6I0xxiX/+cJRntvfNvWOM2RFb4wxLujoH+Fk/wgVRRlhfy8remOMccHBlj4AKoqt6I0xJiYdcIp+uR3RG2NMbDrQ0ktOWiL56Ulhfy8remOMccHBlj6WF2bwJku+h4wVvTHGRJjfrxxq7Y/I+DxY0RtjTMTVdQ4yNOaLyIwbsKI3xpiIO9DSC8DyIm9E3s+K3hhjIuxASx8isKwwPSLvZ0VvjDERdrCljwU5qaQmTvu23TNiRW+MMRF2sKUvIvPnJ0znnrGbRKRNRPYEbfupiOxwHrUTtxgUkYUiMhT0te+HM7wxxsw2Q6M+ajsGIjY+D9O7Z+wDwPeAhyY2qOqtE5+LyDeBnqD9j6pqVagCGmNMLDnc1odfYUUEj+inc8/YzSKycLKvSWCm/y3AW0MbyxhjYlMklz6YMNMx+suBVlU9HLRtkYhsF5EXReTyGX7/NzXu83OotY+BkfFwvo0xxoTMwZY+khM8LMhNi9h7zrTobwceCXreDMxX1XXAZ4CfiMikA1EislFEakSkpr29/ZzefFtdN2/79ma2HO88p9cbY0ykHWzpo7wggzhP+Jc+mHDORS8i8cC7gJ9ObFPVEVXtcD7fChwFlk32elW9T1WrVbU6Pz//nDJM/Omz37n4wBhjot2Blt6IDtvAzI7orwEOqGrDxAYRyReROOfzxUA5cGxmEc8sMyWBeVkpHGjuC9dbGGNMyLT1DnOyf5TK4sjNuIHpTa98BHgVWC4iDSLyYedLt/HnwzYAVwC7RGQn8HPg46oa1nGVFcUZpy4nNsaYaLa3OdBVK0siW/TTmXVz+xm2f2CSbY8Dj8881vRVFHl5/mA7I+M+kuLjIvnWxhhzVvY1BYp+RYSLftZfGVtRnIHPrxxp63c7ijHGvKl9Tb3Mz0nFm5wQ0fed/UXvXF1m4/TGmGi3t6kn4uPzEANFvzA3laR4j43TG2OiWv/IOLUdgxEfn4cYKPr4OA/LCjPYb0f0xpgott85EVtpRX9uKops5o0xJrpNnIhdWZIZ8feOjaIv9nKyf5T2vhG3oxhjzKT2NfWSk5ZIoTcp4u8dE0U/sQqcHdUbY6LV3uYeVpZ4CawFGVkxUfQVxTbzxhgTvcZ8fg619Lsy4wZipOgn/hyyNW+MMdHoSFs/oz6/KydiIUaKHgLz6e2I3hgTjf50ItaKfkYqijM40tbPmM/vdhRjjPkze5t6SUmIY1FeuivvHzNFv6LIy6jPz7H2AbejGGPMn9nX3ENFcWTXoA8WM0VfUeysTd9s4/TGmOihquxr6nXtRCzEUNEvyU8nMd7DPit6Y0wUqescpHd4nFXzIn+h1ISYKfqEOA8VRRnsaexxO4oxxpyyqyHQSaut6ENjZYmXvU29qKrbUYwxBoDdjT0kxnsifvvAYNO5w9QmEWkTkT1B2+4RkUYR2eE8bgj62t0ickREDorIdeEKPpmVJZn0DI3R0DUUybc1xpgz2tXQTWWxl4Q4946rp/PODwAbJtn+bVWtch7PAIhIJYFbDK50XvOfE/eQjYSJOap7m2yc3hjjPr9f2dPYy5pS94ZtYBpFr6qbgene9/Um4FFVHVHV48ARYP0M8p2VFcVe4jzC3iYbpzfGuO94xwD9I+Oujs/DzMboPyEiu5yhnWxn2zygPmifBmdbRCQnxLEkP82O6I0xUWG3cyJ2TWmWqznOtejvBZYAVUAz8E1n+2RXA0x6ZlRENopIjYjUtLe3n2OMv7SqJNOO6I0xUWFXQw8pzgGom86p6FW1VVV9quoHfsifhmcagLKgXUuBpjN8j/tUtVpVq/Pz888lxqQqS7y09o7Y2vTGGNftbuxmZYmXeBdPxMI5Fr2IFAc9vRmYmJHzS+A2EUkSkUVAObBlZhHPzsRFCXZUb4xxk885Ebva5ROxAPFT7SAijwBXAnki0gB8GbhSRKoIDMvUAh8DUNW9IvIYsA8YB+5SVV94ok+uMmjmzZXLCyL51sYYc8rR9n6Gxnyuz7iBaRS9qt4+yeb732T/rwJfnUmomfAmJ7AgN9WO6I0xrvrTFbHunoiFGLsydsLKEi97Gm3mjTHGPbsbuklLjGNxnrsnYiFmiz6Tus5BeobG3I5ijJmjdjb0sGpeJh6XliYOFqNFHxin32fz6Y0xLhjz+dnX7P4VsRNisuht5o0xxk0HW/oYHfez2uULpSbEZNHnpSdRkpnMzgYremNM5G2v6wLgvPlW9GG1pjSLXQ3dbscwxsxB2+u6yc9IYl5WittRgBgu+rVlWZzoGKR7cNTtKMaYOWZ7fTfryrIQcf9ELMRy0TsnQXbZ8I0xJoK6BkY5fnKAdfOzp945QmK26Fc5Rb+z3oZvjDGRs8PpnHVRMj4PMVz03uQEFuen2QlZY0xEba/rwiNEzdRKiOGiB1hbmsXOhm67h6wxJmK213dTUeQlNXHKFWYiJqaLfk1pJu19I7T0DrsdxRgzB/j9yo667qgatoEYL/q1ZYEf9s56G74xxoTf0fZ++kbGo+pELMR40VcWe4n3iM2nN8ZExPa66DsRCzFe9MkJcSwvyrAplsaYiNhe30VmSgKLct1fsTJYTBc9BK6Q3dnQjd9vJ2SNMeG1va6bqrKsqFixMljMF/3a0kz6hsep7RhwO4oxJob1j4xzsLUv6oZtYBpFLyKbRKRNRPYEbfuGiBwQkV0i8qSIZDnbF4rIkIjscB7fD2f46VjjrB5nwzfGmHDaXteFKlF3Ihamd0T/ALDhtG3PAqtUdQ1wCLg76GtHVbXKeXw8NDHP3bLCdFIS4k5drWaMMeHwRm3gQqloWbEy2JRFr6qbgc7Ttv1OVcedp68BpWHIFhLxcR7WlGayzVk21BhjwqGmtpPKEi8ZyQluR/kLoRij/xDw66Dni0Rku4i8KCKXh+D7z9h5C7LZ19TL8JjP7SjGmBg05vOzva6b6gU5bkeZ1IyKXkS+BIwDDzubmoH5qroO+AzwExHxnuG1G0WkRkRq2tvbZxJjSufPz2bcrzZOb4wJi71NvQyN+Vi/KMaKXkTuBN4BvE+dxWRUdURVO5zPtwJHgWWTvV5V71PValWtzs/PP9cY0zJxFnzrCRu+McaE3hvHA6Pb1Qui70QsnGPRi8gG4PPAjao6GLQ9X0TinM8XA+XAsVAEnYnc9CQW5qbaOL0xJizeqO1kQW4qBd5kt6NMajrTKx8BXgWWi0iDiHwY+B6QATx72jTKK4BdIrIT+DnwcVXtnPQbR9h5C7Kd6U924ZQxJnRUlZoTXVywMDqHbQCmXEdTVW+fZPP9Z9j3ceDxmYYKh/PmZ/PEtkbqOgdZEGWXJxtjZq+j7QN0DoxywcLoHLaBOXBl7ITznIsYbPjGGBNKNbXO+HwUH9HPmaJfXpRBWmIc207YhVPGmNDZUttJbloii/Oid6RgzhR9nEeomp9lR/TGmJCqqe2iemE2ItG1kFmwOVP0EBi+2d/cy8DI+NQ7G2PMFFp7h6nrHIzqE7Ew14p+QTZ+hZ12IxJjTAi8dqwDIGovlJowt4q+zDkhaxdOGWNC4LVjHWQkx7OyJNPtKG9qThV9ZmoC5QXp1FjRG2NC4NWjHVy4KIe4KLvRyOnmVNFD4E+smtoufHbHKWPMDDT3DFHbMchFi3PdjjKlOVn0/SPj7G/udTuKMWYWe/VoYHz+4iVW9FFn4qTJ68ejYmUGY8ws9crRDrJSE1hRNOkCvVFlzhV9cWYK83NS2XK8w+0oxphZ7NWjHVy0KDfqbgQ+mTlX9BA4qn+j1hY4M8acm/rOQRq7h2bFsA3M4aLvHBjlaHu/21GMMbPQbBqfhzla9BfaOL0xZgZePdZBXnoi5QXpbkeZljlZ9PNzUinISGKLFb0x5iypamB8fnFuVK9vE2xOFr2IsH5RDq8f67RxemPMWTl+coCW3uFZM2wD0yx6EdkkIm0isidoW46IPCsih52P2c52EZF/E5EjIrJLRM4LV/iZuHBRDi29wzR0DbkdxRgzi7zsjM9fsiTP5STTN90j+geADadt+wLwnKqWA885zwGuJ3Cv2HJgI3DvzGOG3vpFgd/GNk5vjDkbmw+1U5qdwsLcVLejTNu0il5VNwOnN+JNwIPO5w8C7wza/pAGvAZkiUhxKMKGUnlBOlmpCbx+zObTG2OmZ8zn59WjHVyxLH/WjM/DzMboC1W1GcD5WOBsnwfUB+3X4GyLKh6PcOGiHF452mHj9MaYadle103/yDhXlOe7HeWshONk7GS/5v6iSUVko4jUiEhNe3t7GGJM7dKleTR2D1HXOejK+xtjZpfNh9qJ8wiXLJ09J2JhZkXfOjEk43xsc7Y3AGVB+5UCTae/WFXvU9VqVa3Oz3fnt+PEyZSXj9jwjTFman883M66siy8yQluRzkrMyn6XwJ3Op/fCfwiaPv7ndk3FwE9E0M80WZJfhqF3iRePnrS7SjGmCjXOTDKrsYeLp9lwzYA8dPZSUQeAa4E8kSkAfgy8HXgMRH5MFAHvMfZ/RngBuAIMAh8MMSZQ0ZEuHRJHi8casfv11mxOJExxh0vHzmJKlyxbPZMq5wwraJX1dvP8KWrJ9lXgbtmEiqSLlmaxxPbGznQ0kdlSfQvN2qMccfmQ+1kpiSwpjTL7ShnbU5eGRvsUuekyis2fGOMOQNVZfPhdi5bmhf1tw2czJwv+uLMFBbnpfHSESt6Y8zkDrf109o7wuXls2/YBqzoAbhkaS5bjncyOu53O4oxJgq9cDAwqfCKZbPvRCxY0QNw6ZI8Bkd97GzodjuKMSYK/X5/GxVFGZRkpbgd5ZxY0RO4eYBI4Ky6McYE6x4cZeuJLq5ZUeh2lHNmRQ9kpSayqiSTlw5b0Rtj/tyLh9rx+ZWrVxRMvXOUsqJ3XLEsj+313fQMjbkdxRgTRZ7b30ZeeiJrZ+G0yglW9I4rlxfg86sN3xhjThnz+XnhYBtXLS+Y1RdUWtE71pVlkZEcf+rsujHGbD3RRe/w+KwetgEr+lPi4zxcXp7Hi4fabdliYwwAz+1vJTHOMyvXtwlmRR/kymUFtPaOcKClz+0oxpgo8NyBNi5akkta0rRWi4laVvRBJi6GePGQO+vjG2Oix/GTAxxrH+Dqitk9bANW9H+mKDOZiqIMG6c3xvD7fa0AvNWKPvZcubyAmtou+oZtmqUxc9mv9zSzssRLWc7suQn4mVjRn+bK5fmM+5VXjtpdp4yZq1p6htlW1831q4rcjhISVvSnOX9BNulJ8bxw0MbpjZmrfru3BYANq4pdThIaVvSnSYjzcNnSPJ4/0GbTLI2Zo57Z3cyywnSWFqS7HSUkzrnoRWS5iOwIevSKyKdE5B4RaQzafkMoA0fCNZWFtPQOs6ex1+0oxpgIa+8b4Y3azpg5mocZFL2qHlTVKlWtAs4ncH/YJ50vf3via6r6TCiCRtJbKwrwCDy7r8XtKMaYCPvdvhb8SsyMz0Pohm6uBo6q6okQfT9X5aQlUr0wh98506uMMXPHb/a0sCgvjYqiDLejhEyoiv424JGg558QkV0isklEskP0HhH1tspCDrT0Ud856HYUY0yEdA2M8srRDjasKkJk9i5idroZF72IJAI3Aj9zNt0LLAGqgGbgm2d43UYRqRGRmvb26Jvhcm1l4CYDdlRvzNzx7P5WfH6NqWEbCM0R/fXANlVtBVDVVlX1qaof+CGwfrIXqep9qlqtqtX5+dG3YNCC3DSWF2bYOL0xc8jTO5soy0lh9bxMt6OEVCiK/naChm1EJPhU9c3AnhC8hyuurSzkjdouugZG3Y5ijAmztr5hXj5ykpvWzoupYRuYYdGLSCpwLfBE0OZ/FZHdIrILuAr49Ezew03XVhbi8yt/OGBr3xgT657e2Yxf4Z3rStyOEnIzWntTVQeB3NO23TGjRFFk9bxMCr1JPLuvlXefX+p2HGNMGP1iRyMrS7wsLYid2TYT7MrYN+HxCNdWFvLioXYGR8fdjmOMCZNj7f3saujhnVXz3I4SFlb0U3j76hKGxnw8fyD6ZgYZY0LjqR1NiMBfrY29YRuwop/S+kU55Gck8atdTW5HMcaEgaryix2NXLw4l6LMZLfjhIUV/RTiPMINq4r4w4E2BkZs+MaYWLOjvpsTHYMxO2wDVvTT8vY1JYyM+3nOZt8YE3Oe2t5IYryHDatj6yKpYFb001C9IJtCbxK/2mnDN8bEkuExH0/taOK6lUV4kxPcjhM2VvTT4PEIN6wu5oVD7XaLQWNiyG/3ttAzNMZtF5S5HSWsrOin6R1rihkd9/Pcfhu+MSZW/PSNespyUrh4ce7UO89iVvTTtK4sm+LMZJt9Y0yMONExwCtHO7jl/DI8ntha8uB0VvTT5PEIb19dzIuH2uketLVvjJntflbTgEfgf1XH/lXvVvRn4ebz5jHmU562k7LGzGrjPj8/21rPlcsLKM5McTtO2FnRn4XKYi8VRRn8fFuj21GMMTPw4qF2WntHuDXGT8JOsKI/CyLCu88rZWd9N0fa+t2OY4w5R49sqSMvPYm3VhS4HSUirOjP0k1VJXgEntjW4HYUY8w5qOsY5LkDbdy+voyEuLlRgXPjnzKECrzJXLEsnye3N+Lzq9txjDFn6cev1RInwvsuXOB2lIixoj8H7z6vlOaeYV471uF2FGPMWRgcHeenb9SzYVVRzC5gNplQ3By81rmj1A4RqXG25YjIsyJy2PmYPfOo0ePaykIykuN5fKsN3xgzmzy5vZHe4XE+cMlCt6NEVKiO6K9S1SpVrXaefwF4TlXLgeec5zEjOSGOd6wp5td7WmxJBGNmCVXlwVdqWVni5fwFMXXsOaVwDd3cBDzofP4g8M4wvY9rbr1gPkPOgkjGmOj36tEODrX284FLFsbczb+nEoqiV+B3IrJVRDY62wpVtRnA+Rhzc5jWlmayssTLw6+dQNVOyhoT7Ta9XEtOWmLM3kXqzYSi6C9V1fOA64G7ROSK6bxIRDaKSI2I1LS3z77b9Ilz1v5ASx/b6rrdjmOMeROHW/v4/f5W/vqiBSQnxLkdJ+JmXPSq2uR8bAOeBNYDrSJSDOB8/IslH1X1PlWtVtXq/Pz8mcZwxY1VJaQnxfOT1+vcjmKMeRP3vniUlIS4OXcSdsKMil5E0kQkY+Jz4G3AHuCXwJ3ObncCv5jJ+0Sr9KR43rmuhF/tarKFzoyJUg1dg/xyRxO3rS8jJy3R7TiumOkRfSHwkojsBLYA/6OqvwG+DlwrIoeBa53nMem96xcwMu7ncVv/xpio9KM/HkcEPnr5YrejuCZ+Ji9W1WPA2km2dwBXz+R7zxaVJV7Om5/Fw6+f4EOXzr2z+cZEs47+ER59o453Vs2jJCv2V6k8E7syNgTed+ECjrUP8MfDJ92OYowJ8sArtYyM+/nYW5a4HcVVVvQh8I61xeRnJPGjl467HcUY4+geHOWBl2u5rrKIpQXpbsdxlRV9CCTFB87mbz7UzsGWPrfjGGOA7794jP7RcT597TK3o7jOij5E3rt+PskJHu5/6ZjbUYyZ89p6h3nglePctLaE5UUZbsdxnRV9iGSnJfKe88t4ansTbX3DbscxZk77j+ePMOZTPnWNHc2DFX1IfeiyRYz5/fz41RNuRzFmzqrvHOQnW+q4pbqMhXlpbseJClb0IbQoL41rVhTy36+dYGjU53YcY+ak7z53GBHh765e6naUqGFFH2Ifu2IxXYNj/GSLLYtgTKTtaezh8W0NvP+iBRRnzt1586ezog+x6oU5XLw4l++/eJThMTuqNyZSVJV/enof2amJ/O3V5W7HiSpW9GHwyWvKae8bscXOjImgX+1qZkttJ3//tuVkpiS4HSeqWNGHwUWLc7lwUY4d1RsTIUOjPr72zH4qi73cekGZ23GijhV9mHzymnLa+kZ41MbqjQm7H2w+SlPPMPfcuJI4j603dTor+jC5eHEu6xfmcK8d1RsTVrUnB7j3haO8fU0x6xfluB0nKlnRh4mI8KlrymntHbF59caEid+vfP7xXSTGe/jHd1S6HSdqWdGH0SVL87hiWT7//ofDdmMSY8Lg0Tfqef14J//n7Sso9Ca7HSdqWdGH2d3XV9A3Ms73/nDE7SjGxJTmniG+9sx+LlmSyy3VdgL2zVjRh9mKYi/vOb+Uh149QV3HoNtxjIkJqso/PLWHMb+fr71rtd3wZwrnXPQiUiYiz4vIfhHZKyKfdLbfIyKNIrLDedwQuriz02euXY7HA//62wNuRzEmJjz6Rj2/39/G379tOQtybT2bqczkiH4c+KyqrgAuAu4SkYmzId9W1Srn8cyMU85yRZnJbLx8Mb/a1cy2ui634xgzqx1u7eMrT+/l8vI8PnTpIrfjzArnXPSq2qyq25zP+4D9wLxQBYs1G9+yhCJvMv/nyT2M+/xuxzFmVhoe8/G3j2wnLTGeb96yFo/NmZ+WkIzRi8hCYB3wurPpEyKyS0Q2iUh2KN5jtktPiufLf1XJvuZeHnil1u04xsxK//LMfg609PHNW9ZSkGGzbKZrxkUvIunA48CnVLUXuBdYAlQBzcA3z/C6jSJSIyI17e3tM40xK2xYVcRVy/P51rOHaOoecjuOMbPKU9sbeejVE3zkskVcubzA7TizyoyKXkQSCJT8w6r6BICqtqqqT1X9wA+B9ZO9VlXvU9VqVa3Oz8+fSYxZQ0T4p5tW4XdW2TPGTM/O+m4+9/guLlyUw+evr3A7zqwzk1k3AtwP7FfVbwVtLw7a7WZgz7nHiz1lOan83dXl/GZvC7/b2+J2HGOiXmvvMBt/XENBRhL3/vX5JMTZrPCzNZOf2KXAHcBbT5tK+a8isltEdgFXAZ8ORdBY8pHLFrOi2MsXn9zNyf4Rt+MYE7WGRn1s/PFW+obH+dGd1eSkJbodaVaayaybl1RVVHVN8FRKVb1DVVc7229U1eZQBo4FifEevnNrFb1D49z9xG5U1e1IxkSd0XE/f/PwVnY1dPPtW6uoKPK6HWnWsr+BXLK8KIPPbVjOs/taeaym3u04xkQVn1/5zGM7eP5gO/9y82quW1nkdqRZzYreRR+6dBEXL87lK0/v40THgNtxjIkKqso//GIPv9rVzN3XV3D7+vluR5r1rOhd5PEI/++WtcR5hL95eJutW2/mPJ9f+dJTe/jJ63X8zZVL+NhblrgdKSZY0btsXlYK37m1ir1NvXzxSRuvN3PX6LifTz66/VTJ/+/rlrsdKWZY0UeBq1cU8qlrynliWyP//ZrdpMTMPYOj42z8cc2p4ZrPbaiwFSlDKN7tACbg795azu6GHr7y9D4qir1csNBuiWbmhvrOQTb+eCsHWnr52rtW25h8GNgRfZTweIRv3VpFWU4qH32ohiNtfW5HMibsXjlykhu/9xINXYP81wcusJIPEyv6KJKZksCDH1xPvEd4//1baOkZdjuSMWEx7vPzH88f4Y5NW8hNT+KXn7jM1q8JIyv6KDM/N5UHPrienqEx7ty0hZ7BMbcjGRNStScHuOUHr/KN3x7k+lVFPHXXpSzKs5uHhJMVfRRaNS+TH9xRzbGT/dz5X1voGbKyN7PfmM/Pj/54jOu/+0eOtPXz3duq+N57zyM9yU4VhpsVfZS6rDyP7733PPY29fDeH75G58Co25GMOWebD7Wz4Tub+b//s5+LFufw209fwU1Vdp+iSLGij2LXrSzih++v5khbP7fd9yptfTZmb2aXLcc7ueP+13n/pi34/Mr9d1az6QMXUJyZ4na0OUWi4QKd6upqrampcTtG1HrlyEk+8lANOWmJ/OjOalvcyUQ1n1954WAbP9h8jC3HO8lLT2TjFYu585KFJMXHuR0vpojIVlWtnnI/K/rZYWd9Nx99qIaBkXG+c9s6rq0sdDtSzFBVeobGaO8b4WT/KF2DgUf34BgDI+P0O4+RcT8jY35GfX78fsWvgYdHJPDwCIlxHpISPCTFe0hNjCMtMZ7UxHi8KfF4kxPITEkgOy2B7NREctISyUxJiJkLg+o7B/n51gYeq6mnuWeYIm8yH3vLYm67YD4piVbw4WBFH4NaegI3YNjd2MOnr1nGXVctJc5ujjwlVaW9f4QTHYOc6BikrmOAhu4hGruGaOoZorV3hNHxyW/YHu8R0pPjSUuMdwo8jsR4D3ECcR5BEBTFrzDuV8bG/YyM+xge8zM05mPA+QVxJvEeIS+3HgPfAAAJtklEQVQ9ibyMRAoykin0JpHvfCzyJlPoTaY4M5mctMSo+4Xg8yv7mnr5w4E2frO3hf3NvYjAFeX53L6+jKtXFNpNQsLMij5GDY/5uPuJ3Ty5vZELFmbzrVsCF1mZQKG39o5woKWXQ619HGzp50h7P8fa++kbHj+1n0egyJvMvOwUSrJSKPImk5+RRIE3mbz0wJF2dmrgaDsp3jPjgh3z+ekfHqdnaIyeobFTfzF09I/SMTDKyb4R2vtHaO8bobV3hI6BEU7/3zIx3kNxZjJF3mSKMp2HN/Ao8AZ+MeSlJ5GcEJ4jZ1WlqWeYA8297G/uZeuJLmpqu+gbGUcEzp+fzXUri7h+dRGl2fbfY6RY0ccwVeWpHY3841N78avyD++o5JbqMjxz6Oje71eOdwywp7GHPY097GvuZV9TL11B1x3kZyRRXpDOkvx0FuensSgvjQW5aczLSiExPnqPNMd8fk72j9DSMxx49AY+Np/2fNT3l38pZKYkkJeeSG5aEtlpCeSkJeJNTsCbkkBGcjzJCXGBR7zHGW4KvG50XBn1+RkZ89EzNEb3YOAXUnPPMI1dQzR0DTIw+qfVVZfkp3Hh4lwuXJTDxUtyKchIjtSPxwRxvehFZAPwXSAO+JGqfv1M+1rRn5uGrkE++9hOXj/eydqyLO75q0rWzc92O1bIqSqN3UPsrO9hV0M3O+q72dPYc6p4EuM9VBRlUFnsZUWxl4qiDJYVZpAdw7edU1U6B0Zp6R2mrXeElt7hU38ZnOwfoXNg1HmM0Tc89qbDR5PxCGSlJp76y2deVgpLCtKpLA78bDOSE8L0T2bOhqtFLyJxwCHgWqABeAO4XVX3Tba/Ff258/uVJ7c38vXfHKC9b4Sb183jrquWsrQg3e1o56x/ZJxd9d1sr+9me10XO+p7Tt1bNzHOw4oSL2tLM1k9L5NV8zJZWpBuY8FTGB7z0Tc8zvCY79Q5BL8qqqBAQpyQFB9HUrwncPSfFD+n/kKcraZb9OG6JG09cERVjzlhHgVuAiYtenPuPB7h3eeXct2qIr73hyM88MpxntrRyPWrivj4W5awpjTL7Yhvatzn53BbPzvqu9lR1832+i4Ot/WfGqNenJfGFeV5rJufxdqyLCqKvFE97BKtJoZszNwUrqKfBwTfCLUBuDB4BxHZCGwEmD/fVqybqfSkeL5wfQUfvXwRm14+zkOvnOCZ3S1UFnt5T3UpN1XNI8floQy/X6ntGGB3Yw+7G3rY1dDD7sYehpw7a2WlJlBVlsUNq4tZNz+bqtIsMlNtiMCYmQrX0M17gOtU9SPO8zuA9ar6t5Ptb0M3odc7PMZT2xv5WU0Duxt7iPMI5y/I5qrlBbxlWT7LizLCOjWzZ3CMQ219zuyXPvY1BWZrTIyrJ8V7qCzxUlWWRVVZFmtLs1iQmxp1UwiNiWZuj9FfDNyjqtc5z+8GUNWvTba/FX14HWjp5emdTTx/oJ19zb0ApCXGsWpeJmtKM1mUl86C3FTm56SSm55ISkLcmxauz6/0Do3ROThKa88wTT3DNHcPUdc5SG3HAMdPDp4aU4fAXxuVxV4qS7xUFntZXZpJeUE68TaubsyMuF308QROxl4NNBI4GfteVd072f5W9JHT0jPMy0dOsquhm50NgWmJp18slBjvOTWHPN4jxHmEMZ8yPOYLnNQbGf+Led4ABRlJLMxLY2FuKkvy01lWmEF5YTolmSl2Ys+YMHD1ZKyqjovIJ4DfEpheuelMJW8iqygzmXefX8q7zy8FAkfnzT1D1HUMUt81SOfAGN1Do/QMjjHq8zPuU3x+JSFOSE4IzMoIXMYfuKiowJtESWYKRZnJdrLPmCgVtoWgVfUZ4JlwfX8TGnEeoTQ71a5mNCaG2SCpMcbEOCt6Y4yJcVb0xhgT46zojTEmxlnRG2NMjLOiN8aYGGdFb4wxMc6K3hhjYlxU3GFKRNqBEzP4FnnAyRDFCSXLdXYs19mxXGcnFnMtUNX8qXaKiqKfKRGpmc56D5Fmuc6O5To7luvszOVcNnRjjDExzoreGGNiXKwU/X1uBzgDy3V2LNfZsVxnZ87miokxemOMMWcWK0f0xhhjziAmil5E7hGRRhHZ4TxucDtTMBH5exFREclzOwuAiPyziOxyfla/E5EStzMBiMg3ROSAk+1JEclyOxME7oEsIntFxC8irs/aEJENInJQRI6IyBfczjNBRDaJSJuI7HE7ywQRKROR50Vkv/Pv8JNuZ5ogIskiskVEdjrZvhKu94qJond8W1WrnEfU3PBERMqAa4E6t7ME+YaqrlHVKuBXwD+6HcjxLLBKVdcQuBXl3S7nmbAHeBew2e0gIhIH/AdwPVAJ3C4ile6mOuUBYIPbIU4zDnxWVVcAFwF3RdHPawR4q6quBaqADSJyUTjeKJaKPlp9G/gcEDUnQ1S1N+hpGlGSTVV/p6rjztPXgFI380xQ1f2qetDtHI71wBFVPaaqo8CjwE0uZwJAVTcDnW7nCKaqzaq6zfm8D9gPzHM3VYAG9DtPE5xHWP5fjKWi/4TzJ/8mEcl2OwyAiNwINKrqTreznE5Evioi9cD7iJ4j+mAfAn7tdogoNA+oD3reQJQUV7QTkYXAOuB1d5P8iYjEicgOoA14VlXDki1s94wNNRH5PVA0yZe+BNwL/DOB34b/DHyTQFG4neuLwNsikeN0b5ZLVX+hql8CviQidwOfAL4cDbmcfb5E4E/uhyORabq5ooRMsi0q/iKLZiKSDjwOfOq0v2hdpao+oMo5H/WkiKxS1ZCf45g1Ra+q10xnPxH5IYFx54g4Uy4RWQ0sAnaKCASGIbaJyHpVbXEr1yR+AvwPESr6qXKJyJ3AO4CrNYJzf8/i5+W2BqAs6Hkp0ORSlllBRBIIlPzDqvqE23kmo6rdIvICgXMcIS/6mBi6EZHioKc3E4Yf1NlS1d2qWqCqC1V1IYH/Qc+LRMlPRUTKg57eCBxwK0swEdkAfB64UVUH3c4Tpd4AykVkkYgkArcBv3Q5U9SSwFHW/cB+Vf2W23mCiUj+xMwyEUkBriFM/y/GxAVTIvJjAmetFagFPqaqza6GOo2I1ALVqur66nki8jiwHPATWDX046ra6G4qEJEjQBLQ4Wx6TVU/7mIkAETkZuDfgXygG9ihqte5mOcG4DtAHLBJVb/qVpZgIvIIcCWB1RhbgS+r6v0uZ7oM+COwm8B/7wBfjIaZeSKyBniQwL9HD/CYqv5TWN4rForeGGPMmcXE0I0xxpgzs6I3xpgYZ0VvjDExzoreGGNinBW9McbEOCt6Y4yJcVb0xhgT46zojTEmxv1/LRnM2LjjDr4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2b1b45ee36a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig, ax  = plt.subplots()\n",
    "x = np.linspace(-5, 3, 100)\n",
    "ax.plot(x, f(x))\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the *fmin_bfgs* function to find the minima of a function:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: -3.506641\n",
      "         Iterations: 5\n",
      "         Function evaluations: 24\n",
      "         Gradient evaluations: 8\n",
      "[-2.67298151]\n"
     ]
    }
   ],
   "source": [
    "x_min = optimize.fmin_bfgs(f, -2)\n",
    "print(x_min )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a more detailed introduction to optimization with SciPy see: [Scipy Optimization](http://scipy-lectures.github.com/advanced/mathematical_optimization/index.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Install and Get Started with Scikit-Learn<!--span style=\"color:#0b486b\">4. Install and Get Started with Scikit-Learn</span-->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The scikit-learn version is 0.19.1.\n"
     ]
    }
   ],
   "source": [
    "import sklearn\n",
    "\n",
    "print(\"The scikit-learn version is {}.\".format(sklearn.__version__))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A brief introduction Scikit-Learn<!--4.3 A brief introduction Scikit-Learn-->\n",
    "\n",
    "\n",
    "[Scikit-Learn](http://github.com/scikit-learn/scikit-learn) is a Python package designed to give access to **well-known** machine learning algorithms within Python code, through a **clean, well-thought-out API**. It has been built by hundreds of contributors from around the world, and is used across industry and academia.\n",
    "\n",
    "Scikit-Learn is built upon Python's [NumPy (Numerical Python)](http://numpy.org) and [SciPy (Scientific Python)](http://scipy.org) libraries, which enable efficient in-core numerical and scientific computation within Python. As such, scikit-learn is not specifically designed for extremely large datasets, though there is [some work](https://github.com/ogrisel/parallel_ml_tutorial) in this area.\n",
    "\n",
    "###  Dataset in Scikit-learn <!--span style=\"color:#0b486b\">Dataset in Scikit-learn</span-->\n",
    "\n",
    "Machine learning is about creating models from data: for that reason, we'll start by\n",
    "discussing how data can be represented in order to be understood by the computer.  Along\n",
    "with this, we'll build on our matplotlib examples from the previous section and show some\n",
    "examples of how to visualize data.\n",
    "\n",
    "Most machine learning algorithms implemented in scikit-learn expect data to be stored in a\n",
    "**two-dimensional array or matrix**.  The arrays can be\n",
    "either ``numpy`` arrays, or in some cases ``scipy.sparse`` matrices.\n",
    "The size of the array is expected to be `[n_samples, n_features]`\n",
    "\n",
    "- **n_samples:**   The number of samples: each sample is an item to process (e.g. classify).\n",
    "  A sample can be a document, a picture, a sound, a video, an astronomical object,\n",
    "  a row in database or CSV file,\n",
    "  or whatever you can describe with a fixed set of quantitative traits.\n",
    "- **n_features:**  The number of features or distinct traits that can be used to describe each\n",
    "  item in a quantitative manner.  Features are generally real-valued, but may be boolean or\n",
    "  discrete-valued in some cases.\n",
    "\n",
    "The number of features must be fixed in advance. However it can be very high dimensional\n",
    "(e.g. millions of features) with most of them being zeros for a given sample. This is a case\n",
    "where `scipy.sparse` matrices can be useful, in that they are\n",
    "much more memory-efficient than numpy arrays.\n",
    "\n",
    "Scikit-Learn package include several datasets that you can load and start playing with them. You can consult with the documentation for details of the provided datasets. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "**For example, let's look at the [iris dataset.](http://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html) whose description and historical context can further be found [here](https://en.wikipedia.org/wiki/Iris_flower_data_set)**\n",
    "\n",
    "*<span style=\"color:#0b486b\">First, we use the command `from sklearn import datasets` to import existing datasets which have been prepared in advance from scikit-learn package as well its loading utilities. </span>*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*<span style=\"color:#0b486b\"> The following commands then load the iris dataset, examine its structure, description, data and so forth. You are highly encouraged to spend time to get familiar with the syntax and properties of the data, such as examining the values of 'target_names', 'data', etc. </span>*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['target', 'feature_names', 'DESCR', 'data', 'target_names'])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load iris dataset\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "# examine its keys\n",
    "iris.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iris Plants Database\n",
      "====================\n",
      "\n",
      "Notes\n",
      "-----\n",
      "Data Set Characteristics:\n",
      "    :Number of Instances: 150 (50 in each of three classes)\n",
      "    :Number of Attributes: 4 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      "        - sepal length in cm\n",
      "        - sepal width in cm\n",
      "        - petal length in cm\n",
      "        - petal width in cm\n",
      "        - class:\n",
      "                - Iris-Setosa\n",
      "                - Iris-Versicolour\n",
      "                - Iris-Virginica\n",
      "    :Summary Statistics:\n",
      "\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "                    Min  Max   Mean    SD   Class Correlation\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "    sepal length:   4.3  7.9   5.84   0.83    0.7826\n",
      "    sepal width:    2.0  4.4   3.05   0.43   -0.4194\n",
      "    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\n",
      "    petal width:    0.1  2.5   1.20  0.76     0.9565  (high!)\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: 33.3% for each of 3 classes.\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML iris datasets.\n",
      "http://archive.ics.uci.edu/ml/datasets/Iris\n",
      "\n",
      "The famous Iris database, first used by Sir R.A Fisher\n",
      "\n",
      "This is perhaps the best known database to be found in the\n",
      "pattern recognition literature.  Fisher's paper is a classic in the field and\n",
      "is referenced frequently to this day.  (See Duda & Hart, for example.)  The\n",
      "data set contains 3 classes of 50 instances each, where each class refers to a\n",
      "type of iris plant.  One class is linearly separable from the other 2; the\n",
      "latter are NOT linearly separable from each other.\n",
      "\n",
      "References\n",
      "----------\n",
      "   - Fisher,R.A. \"The use of multiple measurements in taxonomic problems\"\n",
      "     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\n",
      "     Mathematical Statistics\" (John Wiley, NY, 1950).\n",
      "   - Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis.\n",
      "     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\n",
      "   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\n",
      "     Structure and Classification Rule for Recognition in Partially Exposed\n",
      "     Environments\".  IEEE Transactions on Pattern Analysis and Machine\n",
      "     Intelligence, Vol. PAMI-2, No. 1, 67-71.\n",
      "   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\n",
      "     on Information Theory, May 1972, 431-433.\n",
      "   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\n",
      "     conceptual clustering system finds 3 classes in the data.\n",
      "   - Many, many more ...\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# print out its description\n",
    "print(iris['DESCR'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(150, 4)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = iris['data']\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Further reading \n",
    "* More tutorials, videos on machine learning with scikit-learn can be found at [Introduction to machine learning with scikit-learn](https://github.com/justmarkham/scikit-learn-videos).\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
